Towards real-time, machine learning enhanced signal detection in 5G-NR

Mobile communication technology is essential in our connected society. In 2022, the deployment of the 5th Generation New Radio (5G-NR) standard is well on its way in many countries. 5G-NR supports new use cases and targets improvements in critical metrics such as latency, peak throughput, reliability, spectral efficiency, and more. Signal Detection - the process of recovering sent symbols from noisy observations - is one of the core tasks in the physical layer of 5G-NR. In recent years, Machine Learning (ML) has been proposed to achieve higher detection performance on detection problems such as Orthogonal Frequency-Division Multiplexing (OFDM) and massive Multiple-Input Multiple Output (MIMO). Since it is impossible to model and design all aspects of the communication link perfectly, the idea is to allow the detection algorithms to be flexible and learn from previous observations. This can mean learning from previous detections while deployed in the field (online learning) or having been trained with example detection problems before deployment (offline training). Many proposals have been made in the literature, showing these performance gains compared to classical detectors; however, the scalability and deployability of these algorithms are typically not considered due to the recent emergence of this field. This thesis takes steps towards closing this gap by developing real-time capable machine learning detectors for OFDM and massive MIMO. The real-time property is characterised by the timely computation of the detection task. The allowed maximum time is determined by protocol standards and system design considerations and is typically in the range of tens to a few hundred microseconds. In this work, we first identify the most promising ML detection algorithms, and their detection performance is verified in link-level simulations. Due to the high computational complexity and the low latency requirement of the detection process, custom hardware design is required to realise real-time processing. For this reason, the algorithms are optimised for Field Programmable Gate Array (FPGA) deployment, and respective hardware architectures are proposed. The resulting digital circuits are synthesised for the Xilinx Ultrascale+ RFSoC and profiled via digital simulation. For the OFDM detection process, a model-driven detector consisting of fully connected neural network layers is investigated. It performs joint channel estimation from pilot symbols and signal detection. The detector structure is optimised based on previous work, reducing the memory requirement by approximately four times. As on-chip memory storage and off-chip memory bandwidth are the performance limiting factors, deep compression techniques (i.e. pruning, quantisation, and Huffman coding of the weights) are investigated. Applying these techniques, the detector achieves a memory reduction of approximately 27 times as compared to previous work. Based on this, an FPGA accelerator is proposed featuring batch-processing, efficient intermediate result buffering and superscalar Huffman decompression circuits. The synthesis and simulation results show that the targeted FPGA platform can process up to 832 sub-carriers in real-time at an average detection latency of approximately 3.7 symbols (246.6 μs). This design shows the feasibility of processing OFDM detection in real-time; however, it also highlights current limitations, such as poor scalability to high-order modulation schemes and the still high memory requirement. For massive MIMO, various ML-enhanced detectors using Orthogonal Approximate Message Passing (OAMP) have been proposed in the literature. These detectors are typically computationally very complex (e.g. OAMPNet) or require online training on the current channel realisation, making practical deployment difficult (e.g. MMNet). A new detector named LAMANet is proposed, which reduces computational complexity by using the simpler Approximate Message Passing (AMP) algorithm without losing performance. Further, the online training requirement is relaxed by incorporating learnable parameters in a new way and by providing precomputed initialisation matrices to the detection algorithm. Finally, A deeply pipelined FPGA architecture is proposed for LAMANet to maximise throughput. The accelerator is synthesised, simulated, and profiled for many antenna configurations and modulation types. The pipelined latency is between approximately 0.3 μs for an arrangement of 32 base station antennas and eight connected users and approximately 4 μs for a configuration of 128 base station antennas and 64 connected users. LAMANet reaches the same throughput of approximately 8 Mbit/s for a configuration of 128 base station antennas and eight connected users as a classical AMP accelerator. However, the core advantage of LAMANet is the ML enhancement which allows it to outperform classical AMP approaches vastly in terms of detection performance on realistic channels